Evolutionary Dynamics in Complex Networks of Competing Boolean Agents 
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We investigate the dynamics of network minority games on Kauffman's NK networks (Kauffman 
nets), growing directed networks (GDNets), as well as growing directed networks with a small 
fraction of link reversals (GDRNets) . We show that the dynamics and the associated phase structure 
of the game depend crucially on the structure of the underlying network. The dynamics on GDNets 
is very stable for all values of the connection number K, in contrast to the dynamics on Kauffman's 
NK networks, which becomes chaotic when K > K c — 2. The dynamics of GDRNets, on the 
other hand, is near critical. Under a simple evolutionary scheme, the network system with a "near" 
critical dynamics evolves to a high level of global coordination among its agents. In particular, the 
performance of the system is close to the optimum for the GDRNets; this suggests that criticality 
leads to the best performance. For Kauffman nets with K > 3, the evolutionary scheme has no 
effect on the dynamics (it remains chaotic) and the performance of the MG resembles that of a 
random choice game (RCG). 
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I. INTRODUCTION 

Complex networks have attracted immense interest in 
recent years, due to their great capability and flexibility 
in describing a wide range of natural and social systems. 
The study of the organization of complex networks has 
attracted intensive research interest Q [3- Q ever since 
the seminal works of Strogatz on small-world networks 
P| and Barabasi and Albert [I| on scale-free networks. 
The degree distribution P(k), which is defined as the 
probability of finding a node with exactly k links, has 
been considered as the most important characterization 
of complex networks. The power law or scale-free degree 
distributions have been established in many real-world 
complex networks 0, . 

The dynamics of a complex network can be studied in 
the context of a system of interactive elements (agents) 
on the network; it depends on how the network is or- 
ganized and how the elements interact. In this paper 
we study a network version of the minority game (MG) 
model proposed by Challet and Zhang [||, which is a 
simplification of Arthur's El Farol bar attendance model 
0. The MG model serves as an interesting paradigm 
for a system of adaptive agents competing for limited re- 
sources. The phase structures of the ori gina l MG B and 
the evolutionary version of the game [iA llfj. ITU Il2j| have 
been well understood. Note that in the MG models, the 
agents are not directly linked to one another, but they 
are influenced by the global environment created by the 
collective action of all the agents. 

The study of network dynamics is pioneered by Kauff- 
man 0, ^| who introduced NK random networks and 
studied its Boolean dynamics. Recently there are quite a 
number of studies on different aspects of network dynam- 
ics. Aldana and Cluzel demonstrated that the scale-free 



network favors robust dynamics^J^. Paczuski et al [T^ 
considered the MG model on a random network to study 
the self-organized process which leads to a stationary but 
intermittent state. Galstyan [l^ studied a network MG, 
focusing on how the change of the mean connectivity K of 
a random network affects the global coordination of the 
system of different capacities. Anghel et al [l^ used the 
MG model to investigate how interagent's communica- 
tions across a network lead to a formation of an influence 
network, fn this paper we try to address the question of 
how different network organizations affect the dynamics 
of the system, which has not been fully explored in the 
previous studies. We investigate the dynamics of net- 
works in the context of a network minority game, fn 
particular we would like to know 1) how the dynamics 
and phase structure of the network minority game de- 
pend on the network organization, and 2) how evolution 
affects the dynamics and phase structure of the game. 
We will consider three types of networks: Kauffman's NK 
random networks (Kauffman nets), growing directed net- 
works (GDNnets), and growing directed networks with 
a fraction of link reversals (GDRNets) as described in 
Ref. H3- 

The paper is organized as follows. In the next section, 
we will describe the network MG model and the various 
networks we consider in this paper. Section III presents 
our numerical results and some qualitative analysis. The 
last section is the summary. 



II. NETWORK MINORITY GAME MODEL 

A. MG of network agents 

The network based MG model consists of N (odd 
number) agents described by the state variables Sj = 
{0,1}, i — 1,2, ...,N, each connected to another K 
agents, i\, 12, Ik ■ Each agent has S strategies which 
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are the mapping functions specifying a binary output 
state (0 or 1) for each possible input vector consiting 
of the states of her K connected agents. The state or 
decision of the ith agent at the current time step t is 
determined by the states/decisions of the K agents it 
connects to at the previous time step t — 1, i.e. 

Si(t)=Fi( Sil (t-l),s i2 (t-l),...,s iK (t-l)) (1) 

where Si k (k — 1,2, ...,K) is the state of the fcth agent 
that is connected to agent i, and F?,j — l,--,S are S 
Boolean functions (strategies) taken from the strategy 
space consisting of 2 2 ^ strategies. As in the standard 
minority game, each agent keeps a record of the cumu- 
lative wealth Wi(t) as well as the cumulative (pseudo) 
scores, Qf (i), i — 1,2, N, s = 1, 2, .., S, for each of her 
S strategies. Before the game starts each agent selects 
at random one of her S strategies, and the cumulative 
wealth Wi(t) and strategy scores Qf(t) are initialized to 
zero. At each time step, each agent decides which of 
the two groups (0 or 1) to join based on the best-scoring 
strategy (the strategy that would have made the most 
winning predictions in the past) among her S strate- 
gies. Each agent gains (loses) one point in her cumulative 
wealth for her winning (losing) decision and each strat- 
egy gains (loses) one pseudo-point for its winning (losing) 
prediction. The agents who are among the minority win; 
those among the majority lose. Let A(t) be the number 
of agents choosing 1 at time step t. Then a measure of 
utilization of the limited resources (system performance) 
can be defined as the variance of A(t) over a time period 
T: 

-, t +T 

^=yE^)^) 2 ( 2 ) 

t=t 

where A = y X^tlXf ~ N/2 is the mean number of 
agents choosing 1. Clearly a 1 measures the global coor- 
dination among agents. The optimal (smallest) value of 
variance a 2 is 0.25, where the number of winning agents 
reaches its optimal value, (A — l)/2, in every time step. 
For a random choice game (RCG), where each agent 
makes decision by coin-tossing, the value of the variance 
<7 2 is 0.25A. The game is adaptive as each agent has 
S strategies to choose from, attempting to increase her 
chance of winning. 

It's worthwhile to point out the differences between the 
network MG we discuss here and the original MG defined 
in Ref. 0- In the original MG, the input for each agent's 
strategies at time t is a vector of the winning decisions of 
the game in the previous M time steps. In the network 
(local) MG, however, the input for each agent's strategies 
at time t is a vector consisting of the decisions of the K 
agents she connects to at the previous time step t — 1. 
Thus, the first key difference is that the agents in the 
original MG use global information while the agents in 
the network MG use local information. The second key 
difference is that the original MG is based on the M time- 



step history of global information while the network MG 
employs a one-step forward dynamics. 



B. Evolutionary Minority Game 

We first investigate the adaptive MG models and used 
the results as a basis for comparison with the evolu- 
tionary MG model, which is our main interest. In an 
evolutionary dynamics, the quenched strategies can be 
changed and more generic behaviors will emerge. We use 
the same evolutionary dynamics as the one described in 
Refs. jjj 0|. In this scheme, each agent is required to 
change her S strategies (by choosing S new strategies 
randomly) whenever her cumulative wealth Wi(t) is be- 
low a pre-specified bankruptcy threshold, —W C (W C > 0). 
The bankrupted agents re-set their wealth and strategy 
pseudo-scores to zero, and the game continues. The net- 
work connection, however, does not evolve. We have 
found that this evolution scheme is much more effective 
than the scheme used, for example, in Ref. 0] where 
the evolution happens at the end of every epoch of speci- 
fied duration (say 10, 000 time steps), and only the worst 
performer is required to change her strategies after each 
epoch. 



C. Generation of Different Networks 

The dynamics of the game depends crucially on the 
network organization. In our model, the network con- 
nection, once generated, remains unchanged throughout 
the game. 

The Kauffman NK random network (Kauffman 
net) 

There are a few different ways to generate random net- 
works |l9l |22|. To study dynamics, however, it is more 
convenient to use Kauffman's NK random networks. 
The Kauffman net is generated by specifying A agents 
first, and then connecting each agent randomly to K 
other agents, whose decisions serve as the input to its 
strategies. 

Growing directed network (GDNet) 

A growing directed network can be generated as de- 
scribed in Ref. [20j. We start with an initial cluster 
of K + 1 agents, which are mutually connected (two 
directed links between each pair of agents). At each 
stage, we add a new agent and connect it to K other 
agents already present in the network. The link is 
directed from the new agent to the existing ones, 
meaning that the strategies of the new (younger) agent 
are based on the states of the existing (older) ones. We 
assume that the probability of connecting a new agent 
to an existing one with degree fcj„ is proportional to 
kf n + 1, where ki„ is the number of incoming links to 
the existing agent. The constant 1 is added to give a 
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nonzero starting weight to the agents that have not been 
connected to. For a = 0, we have a growing directed 
random network which we refer to as GDNet I. For 
a > 0, we have preferential attachment. The special case 
of a = 1 corresponds to a scale-free directed network 
which we refer to as GDNet II. In this network, the 
out-degree is K for all the agents, but the in-degree 
follows a power law distribution. The undirected version 
of this model corresponds to the Barabasi- Albert model. 
In the growing networks (random or scale- free), the 
younger agents are influenced by the older ones, except 
for the initial K + 1 agents who are mutually influenced. 
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Growing directed network with a fraction of link 
reversals (GDRNet) 

In order to make our discussion more relevant to many 
real world networks which typically have a fraction of 
feedback links, we here modify the above growing net- 
work to allow a small fraction of link reversals. Let p 
be the probability that each agent has a link reversal: 
when each new agent is connected to other K agents al- 
ready present in the network, each link has a probability 
of q = p/K to have its direction reversed. We consider 
two GDRNcts: GDRNet I with a = and GDRNet II 
with a = l. 

There are two new features for GDRNets that are 
worth mentioning: 1) Some agents may have more than 
K strategy inputs, while others may have fewer than K 
inputs; but the mean number of inputs for an agent re- 
mains as K. 2) Some younger agents can influence the 
older agents. 



FIG. 1: The variance a 2 of the network MG as a function of 
K on Kauffman NK random network. The dash-line in 
the middle is for a 2 — 0.25N (RCG), and the dash-dot line at 
bottom is for a 2 — 0.25 (the theoretical lower bound). The 
system parameters are: S — 2, N — 401 (left plot) and 901 
(right plot). The figures are obtained with 100 simulations 
each. 
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III. NUMERICAL RESULTS AND ANALYSIS 
A. MG on the Kauffman net 

Let's first examine the performance of MG on the 
Kauffman net. The result is shown in Fig. 1. It is clear 
from the figure that when K = 2, the variance a 2 has very 
large fluctuations (four orders of magnitude for N = 401 
and five orders of magnitude for N = 901) with different 
initial conditions. This reflects the fact that the system 
dynamics is critical for K = 2. For K > 3 the system 
performs like a random choice game. The observation we 
obtained here is consistent with the well-known result for 
the Boolean dynamics on Kauffman nets: when K = 2 
the system is at the "edg e of chaos" and for K > 3 the 
system is chaotic [lj, |2lJ . 

The dynamics in the original MG depends on two vari- 
ables: N, the system size and M, the memory size of 
the agents. There are three different phases for differ- 
ent memory value M, described by a Savit curve @. 
The critical value for an optimal global coordination is 
M c ~ ln(iV), which depends on N. For the network MG 
on the Kauffman net, however, the three phases of the 
dynamics are: stable for K = 1, critical for K = 2, and 
chaotic for K > 3. So the dynamics of network MG de- 



FIG. 2: The variance a 2 of the network EMG on the Kauff- 
man net as a function of K. The dash-line in the middle is 
for a 2 = 0.25./V (RCG), and the dash-dot line at bottom is 
for a 2 — 0.25. The system parameters are: S = 2, N — 401 
(left plot) and 901 (right plot), and W c = 1024. The figures 
are obtained with 100 simulations each. 

pends on only one variable, K, and the chaotic regime 
dominates. 



B. EMG on the Kauffman net 

Now let's check how the game performs when the sim- 
ple evolution scheme described above is applied. The 
results are shown in Fig. 2. We see from the figure that 
evolution helps dramatically improve the system perfor- 
mance when the connection number K is samll (K < 3), 
but has virtually no effect for larger K(> 4). This means 
that for K < 3 the system is still at stable or "critical" 
state, but for K > 4 it is chaotic. Note that evolution has 
shifted the critical point from K = 2 to K = 3, suggest- 
ing that it is more powerful than adaptation (modeled 
by strategy switching) in bring out order in complex sys- 
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FIG. 3: The variance a 2 of the network MG as a function of K 
on GDNet I (upper panel) and GDNet II (bottom panel). 
The dash-line in the middle is for a 2 = 0.25iV (RCG), and 
the dash-dot line at bottom is for a 2 — 0.25. The system 
parameters are: S = 2, N = 401 (left plot) and 901 (right 
plot). The figures are obtained with 100 simulations each. 

tems. Since most real- world systems evolve, evolutionary 
dynamical analysis may be more relevant. 

C. MG on Growing Directed Networks 

Let's now examine the performance of MG on growing 
directed networks. Two limiting cases of the growing 
directed networks are checked: 1) GDNet I, the growing 
random directed network (a = 0); 2) GDNet II, the 
growing directed network with preferential attachment 
(a = 1). Fig. 3 shows the results of system performance. 

We have the following observations for the MG dynam- 
ics: 1) there are large fluctuations in the values of the 
variance er 2 ; 2) there seems to be no significant difference 
in the MG dynamics for the two growing network mod- 
els. So in terms of a simple (non-evolutionary) dynamical 
process, all growing networks (irrespective of its value of 
preferential attachment exponent a £ [0,1]) have simi- 
lar dynamics and the scale-free network is not special. 
The stability of the dynamics is due to the construction 
process of growing networks, which leads to a maximum 
state cycle length of 2 K+1 as was pointed out in Ref. [2Cj, 
irrespective of the value of a. 

Comparing the dynamics of growing networks with 
that of the Kauffman net, we see a lot of differences. 
In the Kauffman net (Fig. 1) the dynamics is stable or 
critical for K < 2. In growing networks (Fig. 3), how- 



FIG. 4: The variance a 2 of the network EMG as a function 
of K on GDNet I (upper panel) and GDNet II (bottom 
panel). The dash-line in the middle is for a 2 = 0.25N (RCG), 
and the dash-dot line at bottom is for a 2 = 0.25. The system 
parameters are: S = 2, N = 401 (left plot) and 901 (right 
plot), and W c = 1024. The figures are obtained with 100 
simulations each. 



ever, the dynamics is stable for all the values of K, on 
both GDNet I and GDNet II. Thus non-growing and 
growing networks have very different network dynamics. 



D. EMG on Growing Directed Networks 

To see how evolution affects the dynamics, we plot the 
results on system performance for the EMG on GDNets 
in Fig. 4. By comparing Fig. 3 and Fig. 4, we see that 
evolution helps reduce the variance a 2 dramatically (by 
more than two orders of magnitude). Although the fluc- 
tuations in the variance is still large, the values are all 
below the value of the variance corresponding to RCG. 
We can also see (by comparing panels a-b to panels c-d 
in the Fig.4) that the EMG on GDRNet I performs bet- 
ter than the EMG on GDRNet II, but the difference is 
small. This is not surprising as the essential property of 
all growing networks are the same: the younger agents 
are always influenced by the older ones; and the system 
state cycle is determined only by the initial cluster of 
K + 1 agents, leading to a stable dynamics of the sys- 
tem with the state cycle length limited to the maximum 
length of 2 K+1 . 

By comparing the results for growing networks (shown 
in Figs. 3-4) with the results for the Kauffman net (shown 
in Figs. 1-2), we see that these two types of networks 
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d). EMG on GDRNet-ll (a=1 , p=0.05), N=901 
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FIG. 5: The variance a 2 of the MG as a function of K on 
GDRNet I(upper panel) and GDRNet II (bottom panel). 
The link reversal probability is q = p/K (here p — 0.05 is the 
probability that an agent has a reversed link). The dot-line 
at the top is the average a 2 , the dash- line in the middle is for 
a 2 = 0.257V (RCG), and the dash-dot line at bottom is for 
a 2 — 0.25. The other system parameters are: S — 2, JV = 401 
(left plot) and 901 (right plot). The figures are obtained with 
100 simulations each. 



have very different impacts on the dynamics. The reason 
for this is rooted in the differences in network construc- 
tion process. For the Kauffman net, each agent chooses, 
at random, other K agents for inputs to her strategies. 
Any given agent has a potential to influence many other 
agents. It is not surprising that, for large enough K 
(K > 3), the system is virtually chaotic. However, for 
GDNets (with any value of a £ [0,1]), the dynamics is 
driven by the initial cluster of K + l agents; this results in 
a stable dynamics in which the maximum cycle of length 
is 2 K+1 . 



E. MG/EMG on Growing Networks with Link 
Reversals 

Now we examine the performance of the MG on grow- 
ing networks with link reversals, GDRNet I and GDR- 
Net II. The results for the MG are shown in Fig. 5 and 
the results for the EMG are shown in Fig. 6. 

Fig. 5 shows that, without evolution, the MG on 
GDRNets produces similar results as the MG on 
GDNets. However, the EMG on GDRNets produces 
significantly better results than the EMG on GDNets, 
as shown in Fig. 6. GDRNet II, in particular, gives the 



FIG. 6: The variance a 2 of the EMG as a function of K on 
GDRNet I (upper panel) and GDRNet II (bottom panel). 
The link reversal probability is q = p/K (p — 0.05). The 
dash- line at the top is for a 2 — 0.25/V (RCG), the dot-line 
below is the average a 2 , and the dash-dot line at bottom is 
for a 2 — 0.25. The other system parameters are: W c = 1024, 
S = 2, N = 401 (left plot) and 901 (right plot). The figures 
are obtained with 100 simulations each. 

best performance among all the network models. These 
observations suggest that MG dynamics is not very sen- 
sitive to different attachment algorithm in a growing di- 
rected network. But the attachment algorithm makes 
some difference in the performace of the EMG. If we ex- 
amine the results (shown in Fig. 6) more carefully we see 
that for GDRNet II, the EMG performance is so good 
that the variance a 2 is below 1 most of the time. This 
means that the difference between the attendance num- 
bers in the majority group and the minority group is less 
than 2 on average, which is very close to the theoretical 
bound where the difference is 1 at every time step. 

We have shown in our earlier paper |2£j that, the gen- 
eral Boolean dynamics on GDRNets is close to critical, in 
the sense that the distribution of the state cycle lengths is 
close to a power law. This suggests that, under an evo- 
lutionary dynamics, criticality makes the system more 
efficient. 



IV. SUMMARY 

We have presented an extensive numerical investiga- 
tion on the dynamics of the MG on various networks. 
We have compared the dynamics of the network MG with 
that of the original MG. In the original MG, the critical 
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value for an optimal global coordination is M c ~ ln(iV), 
so the dynamics depends on two variables: N, the sys- 
tem size and M, the memory size of the agents. In the 
network MG, however, the dynamics depends on K only, 
and the system has three phases: stable for K — 1, crit- 
ical for K = 2, and chaotic when K > 3, consistent with 
Kauffman's Boolean dynamics. We have studied the dy- 
namics of the EMG on the Kauffman net, and we have 
found that, it appears to be "critical" for the connection 
number K = 3, which is different from the critical value 
K c = 2 for the non-evolutionary MG. This shows that 
evolution makes a significant difference. 

We have investigated the dynamics of the MG on a gen- 
eral class of growing directed networks. We show that, 
the dynamics on these networks is stable on these grow- 
ing networks, which is very different from the dynamics 
on the Kauffman NK random network. This is due to the 
way the network is constructed. In the Kauffman net all 



the agents are treated equally; every agent has an equal 
probability to influence others. This results in a very 
large "influence network" of a given agent. However, in 
growing directed networks, the initial cluster of agents 
dictate the dynamics of the system; the "junior" agents 
have no influence on the "senior" ones. The maximum 
state cycle length is limited to 2 A+1 . 

We have also studied the MG dynamics for a modi- 
fied growing directed network model which allows a small 
fraction of link reversals. Our numerical results show that 
the best system coordination and performance emerges 
for the EMG on the scale- free network with link reversals 
(GDRNet II); the variance a 2 in the EMG on GDR- 
Net II reaches to such a low level that it's close to the 
theoretical bound most of the time. This suggests that 
evolution makes the agents best coordinated on critical 
scale-free networks. 
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